On Robustness Properties of Beta Encoders and Golden Ratio Encoders
نویسندگان
چکیده
منابع مشابه
Saturating Auto-Encoders
We introduce a simple new regularizer for auto-encoders whose hidden-unit activation functions contain at least one zero-gradient (saturated) region. This regularizer explicitly encourages activations in the saturated region(s) of the corresponding activation function. We call these Saturating Auto-Encoders (SATAE). We show that the saturation regularizer explicitly limits the SATAE’s ability t...
متن کاملSelf-Organising Stochastic Encoders
The processing of mega-dimensional data, such as images, scales linearly with image size only if fixed size processing windows are used. It would be very useful to be able to automate the process of sizing and interconnecting the processing windows. A stochastic encoder that is an extension of the standard Linde-Buzo-Gray vector quantiser, called a stochastic vector quantiser (SVQ), includes th...
متن کاملNeural Semantic Encoders
We present a memory augmented neural network for natural language understanding: Neural Semantic Encoders. NSE is equipped with a novel memory update rule and has a variable sized encoding memory that evolves over time and maintains the understanding of input sequences through read, compose and write operations. NSE can also access multiple and shared memories. In this paper, we demonstrated th...
متن کاملEncoders and Fault Detection
Monitoring large-scale systems is of fundamental importance in modern infrastructures. Many of these large-scale systems are complex interconnections of sub-components which interact by means of communication channels with limited bandwidth. Therefore the information must be encoded in order to be transmitted. Given a class of encoders, the problem of detecting faults affecting some of these su...
متن کاملRate-Distortion Auto-Encoders
A rekindled the interest in auto-encoder algorithms has been spurred by recent work on deep learning. Current efforts have been directed towards effective training of auto-encoder architectures with a large number of coding units. Here, we propose a learning algorithm for auto-encoders based on a rate-distortion objective that minimizes the mutual information between the inputs and the outputs ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Information Theory
سال: 2008
ISSN: 0018-9448
DOI: 10.1109/tit.2008.928235